IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES The Martingale Approach for Concentration and Applications in Information Theory, Communications and Coding

This chapter introduces some concentration inequalities for discrete-time martingales with bounded increments, and it exemplifies some of their potential applications in information theory and related topics. The first part of this chapter introduces briefly discrete-time martingales and the Azuma-Hoeffding & McDiardmid’s inequalities which are widely used in this context. It then derives these refined inequalities, followed by a discussion on their relations to some classical results in probability theory. It also considers a geometric interpretation of some of these inequalities, providing an insight on the inter-connections between them. The second part exemplifies the use of these refined inequalities in the context of hypothesis testing, information theory, communications, and coding. The chapter is concluded with a discussion on some directions for further research. Index Terms Concentration of measures, error exponents, Fisher information, hypothesis testing, information divergence, large deviations, martingales, moderate deviations principle.